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DETAILED ACTION 



Priority 

1 . Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 



2. The prior art rejections are organized in the following manner: 

The apparatus of claim 12 performs the method of claim 1 . The means of the 

apparatus of claim 13 are embodied in the various units disclosed in the apparatus of 

claim 12. The computer program of claim 34 performs the method of claim 1 . 

Therefore, claims 1 , 12, 23 and 34 are discussed together. 

Claims 2-1 1 disclose the same limitations that are listed in claims 13-22 and 

claims 24-33, respectively. Therefore, claims 2-1 1 are discussed with claims 13-22 and 

claims 24-33, respectively. 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Prelude to Prior Art Rejections 



Claim Rejections - 35 USC § 103 
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4. Claims 1-3, 5-6, 12-14, 16-17, 23-25, 27-28 and 34 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Matsuda (US Patent 6,285,470 B1) in view of Fields 
(US Patent 5,767,945). 

Regarding claims 1, 12, 23 and 34: Matsuda discloses an image processing 
apparatus (figure 1 of Matsuda). Figure 2 of Matsuda is a block diagram showing 
details of the control section of said apparatus (column 3, lines 30-31 of Matsuda). 

Said apparatus comprises an edge detection unit (figure 2(21 (part)) of Matsuda) 
which detects an edge in a digital original image (figure 3b("character part") and column 

5, lines 17-21 of Matsuda) obtained by digitally inputting a single side of a document 
printed on both sides of paper (column 3, lines 59-65 of Matsuda). An opened book is 
read by the scanner (figure 1 and column 3, lines 59-65 of Matsuda), so only one side of 
the page is read. A character part is discriminated (column 5, lines 17-21 of Matsuda) 
as is a non-show-through base area (figure 3b("base area") and column 5, lines 25-30 
of Matsuda). Characters are formed from purely black marks, so the character area 
would also correspond to any black edges that are read by the scanner. Furthermore, 
any edges of a grayscale value less than black would also be read by the scanner and 
be classified as part of the base area, since said base area is a separate and 
distinguishable element of the histogram (figure 3b of Matsuda). 

Said apparatus further discloses a background color estimation unit (figure 
2(21 (part)) of Matsuda) which estimates a background grayscale density of said paper 
or a background grayscale image on said single side (column 8, lines 18-24 of Matsuda) 
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with respect to a low-intensity portion (figure 3b(Ls) of Matsuda) of said detected edge 
(figure 3b(Lp,Ls) and column 7, lines 15-22 of Matsuda). 

Said apparatus further discloses an image replacement unit (figure 2(21 (part)) of 
Matsuda) which replaces the low portion of the edge intensity as a component 
corresponding to the show-through with said estimated background grayscale density or 
background grayscale image in said original image (column 8, lines 43-46 of Matsuda). 

Matsuda further discloses that said image replacement unit removes said 
component corresponding to the show-through to generate said show-through removed 
image (column 9, lines 19-23 of Matsuda). 

The show-through compensation operations of the apparatus, among other 
operations, are performed using a CPU (figure 2(21) and column 4, lines 48-53 of 
Matsuda). Therefore, said edge detection unit, said background color estimation unit, 
and said image replacement unit correspond to the portions of said CPU and the related 
physically embodied software that performs the functions of said edge detection unit, 
said background color estimation unit, and said image replacement unit. 

Matsuda does not disclose expressly reading color documents and processing 
color image data. 

Fields discloses scanning in color documents (column 4, lines 47-50 of Fields) 
and processing color image data (column 4, lines 65-67 of Fields). 

Matsuda and Fields are combinable because they are from the same field of 
endeavor, namely processing of image data acquired by an image scanner. At the time 
of the invention, it would have been obvious to a person of ordinary skill in the art to 
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obtain a digital color original image from a color-printed document, detect color density 
instead of grayscale density, and estimate and replace background color instead of 
background grayscale level. The motivation for doing so would have been to be able to 
take advantage of image data derived from color separations and color films (column 4, 
lines 48-52 of Fields). Therefore, it would have been obvious to combine Fields with 
Matsuda to obtain the invention as specified in claims 1, 12, 23 and 34. 

Further regarding claim 34, the operations of said apparatus are implemented 
with a CPU (figure 2(21) and column 4, lines 48-53 of Matsuda) , therefore it is inherent 
that some form of physically embodied software is used. 

Regarding claims 2, 13 and 24: Matsuda discloses that said edge detection 
unit calculates edge intensity from each component of an image in which the edge is 
detected (figure 3b and column 5, lines 25-30 of Matsuda), and detects the edge 
considering a correlation between the respective edge intensity of said components 
(column 5, lines 21-26 of Matsuda). The intensities of each pixel of the image are 
calculated and recorded in a histogram (figure 3b and column 5, lines 17-22 of 
Matsuda). The edges (character portion) are detected based on a correlation between 
the components of the image as shown in the histogram peaks (figure 3b and column 5, 
lines 21 -26 of Matsuda). 

Regarding claims 3, 14 and 25: Matsuda discloses that said edge detection 
unit comprises a binarization unit (figure 2(21 (part)) of Matsuda) which binarizes the 
edge intensity (column 7, lines 59-62 of Matsuda). 
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Matsuda discloses that said background color estimation unit comprises, in edge 
distribution (figures 3a-3c of Matsuda) in which a pixel with the edge intensity higher 
than a threshold (base density) is obtained as an ON pixel and a pixel with the edge 
intensity lower than the threshold is obtained as an OFF pixel (column 8, lines 43-48 of 
Matsuda), a window setting unit (figure 2(21 (part)) of Matsuda) which sets a window 
(figure 3b(Ls->Lp) of Matsuda) of a predetermined size (column 7, lines 28-33 of 
Matsuda) in a region (base area) of said OFF pixels (column 8, lines 43-44 of Matsuda). 
A window is set in the histogram (figures 3a-3c of Matsuda) between the peak 
brightness (Lp) and the base brightness (Ls), as shown in figures 3a-3c of Matsuda, 
said peak brightness and said base brightness defined by a specific relationship 
(column 7, lines 28-33 of Matsuda), and therefore a predetermined size. The base 
region is a region of OFF pixels (column 8, lines 43-44 of Matsuda), contrasted with the 
edge (character) region which has a much higher density (and therefore lower 
brightness) (column 8, lines 38-42 and 45-51 of Matsuda), as is also shown in figure 3b 
of Matsuda. 

Said background color estimation unit further comprises a color clustering unit 
(figure 2(21 (part)) of Matsuda) which classifies the pixels within each of said windows 
set by said window setting unit into two colors, particularly white and darker (column 8, 
lines 43-46 of Matsuda). All of the pixels with a density lower than the base value are 
classified as "white" and all pixels with a density higher than the base value are defined 
in a group corresponding to the ycurve (column 8, lines 48-51 of Matsuda). 
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Said background color estimation unit further comprises an estimation unit (figure 
2(21 (part)) of Matsuda) which estimates a color with higher brightness of said two colors 
as a background color (column 8, lines 43-44 of Matsuda). 

Matsuda further discloses that said image replacement unit comprises a pixel 
replacement unit (figure 2(21 (part)) of Matsuda) which replaces the color of the pixels 
within each of said windows with said estimated background color (column 8, lines 43- 
46 of Matsuda). 

The various operations of the apparatus are performed using a CPU (figure 2(21) 
and column 4, lines 48-53 of Matsuda). Therefore, said binarization unit, said window 
setting unit, said color clustering unit, said estimation unit, and said pixel replacement 
unit correspond to the portions of said CPU and the related physically embodied 
software that performs the functions of said binarization unit, said window setting unit, 
said color clustering unit, said estimation unit, and said pixel replacement unit. 

Regarding claims 5, 16 and 27: Matsuda discloses that said binarization unit 
has an automatic threshold setting unit (figure 2(21 (portion of CPU and related 
embodied software)) of Matsuda) which statistically analyzes the distribution of the edge 
intensity and automatically sets the threshold (column 8, lines 47-51 of Matsuda). Using 
the ycurve, the relationship between the densities is established (column 8, lines 47-51 
of Matsuda). Since the densities are quantized (column 7, lines 60-62 of Matsuda), said 
relationship therefore sets the threshold levels for each grayscale value based on the 
related density that is digitized. 
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Regarding claims 6, 17 and 28: Matsuda discloses that said binarization unit 
has a threshold setting unit (figure 2(21 (portion of CPU and related embodied software)) 
of Matsuda) by which a user sets the threshold according to a degree of show-through 
or paper quality (column 9, lines 1-9 of Matsuda). By selecting the level of base 
brightness, the user also selects the threshold of each of the gray scale levels since the 
base brightness is used to set the zero level (column 8, lines 43-44 of Matsuda) and a 
linear relation is used for the other gray scale levels (column 8, lines 45-51 of Matsuda). 

5. Claims 4, 15 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Matsuda (US Patent 6,285,470 B1 ) in view of Fields (US Patent 5,767,945) and Jin 
(US Patent 5,880,858). 

Regarding claims 4, 15 and 26: Matsuda discloses that said pixel replacement 
unit replaces the color of the pixels determined to be in the background with the 
estimated background color (column 8, lines 43-46 of Matsuda). 

Matsuda further discloses that said background color estimation unit estimates a 
background grayscale density of said paper or a background grayscale image on said 
single side (column 8, lines 18-24 of Matsuda) with respect to a low-intensity portion 
(figure 3b(Ls) of Matsuda) of said detected edge (figure 3b(Lp,Ls) and column 7, lines 
15-22 of Matsuda). 

Matsuda in view of Fields does not disclose expressly a horizontal run formation 
unit which forms a run in each line in the horizontal direction in the region of said OFF 
pixels; and a vertical run formation unit which forms a run in each line in the vertical 
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direction, wherein said background color estimation unit estimates said background 
color or background color image using the horizontal (or vertical) runs formed by said 
horizontal run formation unit (or vertical run formation unit), further estimates a 
background color or a background color image using the vertical (horizontal) runs 
formed by said vertical run formation unit (or horizontal run formation unit) with respect 
to said estimated background color or background color image in the horizontal (or 
vertical) direction, and said pixel replacement unit replaces the pixels of said original 
image corresponding to said OFF pixels with pixels of said estimated background color 
or background color image in the vertical (or horizontal) direction. 

Jin discloses a horizontal run formation unit which forms a run in each line in the 
horizontal direction in the region of OFF pixels (column 3, lines 35-38 of Jin). The OFF 
pixels of the horizontal row are determined since the ON pixels of the horizontal row are 
counted (column 3, lines 35-38 of Jin) and the size of the horizontal row is known 
(column 3, lines 20-23 of Jin). 

Jin further discloses a vertical run formation unit which forms a run in each line in 
the vertical direction (column 3, lines 45-50 of Jin). The OFF pixels of the vertical 
column are determined since the ON pixels of the vertical column are counted (column 
3, lines 45-50 of Jin) and the size of the vertical column is known (column 3, lines 20-23 
of Jin). 

The horizontal and vertical runs are performed by a digital scanner (column 3, 
lines 16-20 of Jin) which must therefore be controlled by some form of CPU. Said 
horizontal run formation unit and said vertical formation unit correspond to the portions 
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of said CPU and the related physically embodied software that performs the functions of 
said horizontal run formation unit and said vertical formation unit. 

Jin further discloses estimating said background region using the horizontal (or 
vertical) runs formed by said horizontal run formation unit (or vertical run formation unit) 
(figure 1b and column 3, lines 35-41 of Jin), further estimating a background region 
using the vertical (horizontal) runs formed by said vertical run formation unit (or 
horizontal run formation unit) with respect to said estimated background region in the 
horizontal (or vertical) direction (figure 1c and column 3, lines 45-52 of Jin). 

Matsuda in view of Fields is combinable with Jin because they are from the same 
field of endeavor, namely image data scanning and processing. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to use 
horizontal runs and vertical run to estimate background characteristics, as taught by Jin, 
said background characteristics being the background color, as taught by Matsuda. 
Therefore, when said pixel replacement unit replaces the color of the pixels determined 
to be in the background with the estimated background color, said replacement will be 
performed with respect to said estimated background region in the horizontal (or 
vertical) direction. The motivation for doing so would have been to remove the 
interference to due background color (column 1 , lines 33-35 of Jin) and be able to 
automatically delineate the non-background region of the image (column 1, lines 28-32 
of Jin). Therefore, it would have been obvious to combine Jin with Matsuda in view of 
Fields to obtain the invention as specified in claims 4, 15 and 26. 
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6. Claims 7-9, 18-20 and 29-31 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Matsuda (US Patent 6,285,470 B1) in view of Fields (US Patent 
5,767,945) and Dhawan (US Patent 5,271,064). 

Regarding claims 7, 18 and 29: Matsuda discloses an edge determination unit 
(figure 2(21 (portion of CPU and related embodied software)) which detects an edge of 
said show-through removed image by said edge detection unit (figure 3b("show-through 
part") and column 5, lines 21-25 of Matsuda), compares said detected edge of the 
show-through removed image and the edge distribution of said original image (column 
5, lines 46-52 of Matsuda), and determines an edge not existing on said original image 
(column 6, lines 30-33 of Matsuda). The edges of the show-through image are detected 
by the histogram (figure 3b("show-through part") and column 5, lines 21-25 of Matsuda) 
wherein the pixel data of said edges are compared with the edges of the original image 
(figure 3b("character part") of Matsuda) and the background of the image (figure 
3b("base area") of Matsuda) based on specific brightness criteria (column 5, lines 46-52 
of Matsuda). The edge not appearing on the original image, and therefore in the show- 
through image, can therefore be determined (column 6, lines 30-33 of Matsuda). 

Matsuda in view of Fields does not disclose expressly a repetition unit which 
generates again a show-through removed image about the periphery of the edge not 
existing said original image in said show-through removed image. 

Dhawan discloses a repetition unit (figure 1(10) of Dhawan) which generates 
again an enhanced image (column 12, lines 10-13 of Dhawan) in order to further 
enhance said image in an iterative fashion (column 12, lines 13-19 of Dhawan). 
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Matsuda in view of Fields is combinable with Dhawan because they are from the 
same field of endeavor, namely image data processing and enhancement. At the time 
of the invention, it would have been obvious to a person of ordinary skill in the art to use 
iterative image enhancement, as taught in Dhawan, to repeatedly update the show- 
through removed image about the periphery of the edge not existing in said original. 
The motivation for doing so would have been to iteratively obtain a maximally enhanced 
image (column 12, lines 3-6 of Dhawan). Therefore, it would have been obvious to 
combine Dhawan with Matsuda in view of Fields to obtain the invention as specified in 
claims 7, 18 and 29. 

Regarding claims 8, 19 and 30: Matsuda discloses that said edge 
determination unit subtracts edge intensity calculated in said original image from edge 
intensity calculated in said show-through removed image at each pixel (column 9, lines 
6-9 of Matsuda), and determines any value higher than a predetermined threshold, in 
said show-through removed image, as an edge not existing in said original image 
(column 8, lines 43-44 and lines 60-62 of Matsuda). The user can select to completely 
subtract the show-through image (column 9, lines 6-9 of Matsuda), which would 
necessarily have to be at each pixel. Any density lower than the detected base density 
is set to white (column 8, lines 43-44 of Matsuda), said base density being detectable 
even under show-through image conditions (column 8, lines 60-62 of Matsuda). If the 
user selects to completely subtract the show-through image (column 9, lines 6-9 of 
Matsuda), then the entire show-through part would therefore be set to the background 
color. 
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Further regarding claims 9, 20 and 31: Dhawan discloses that said repetition 
unit sets the size of said window to a smaller value than that in the previous processing 
when an enhanced image is again generated (column 12, lines 16-25 of Dhawan). 
Image enhancement is performed, and processing time reduced, by iteratively adjusting 
the window (column 12, lines 16-23 of Dhawan). The number of data elements are 
reduced (column 12, lines 23-25 and column 11, lines 60-64 of Dhawan), thus resulting 
in a smaller window. 

7. Claims 10, 21 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Matsuda (US Patent 6,285,470 B1) in view of Fields (US Patent 5,767,945) and 
Saito (US Patent 5,966,455). 

Regarding claims 10, 21 and 32: Matsuda in view of Fields does not disclose 
expressly an image reduction unit which generates a reduced original image with low 
resolution from said original image; a show-through region estimation unit which 
calculates a difference between a show-through removed image generated with respect 
to said reduced original image and said reduced original image to estimate a show- 
through region; and a pixel allocation unit which allocates pixels of said show-through 
removed image, that correspond to said estimated show-through region, onto said 
original image with the original resolution. 

Saito discloses an image reduction unit (figure 12(1106) of Saito) which 
generates a reduced original image with low resolution from said original image (column 
1 0, line 66 to column 1 1 , line 7 of Saito). 
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Saito further disclose a region estimation unit (figure 12(1 107) of Saito) which 
calculates the properties of a region generated with respect to said reduced original 
image (column 1 1 , lines 41-47 of Saito) in order to process said region based on the 
associated properties (column 12, lines 48-53 of Saito). 

Saito further discloses a pixel allocation unit (figure 12(1111) of Saito) which 
allocates pixels of said processed image that correspond to said estimated region 
(column 14, lines 1-7 of Saito), onto said original image with the original resolution 
(column 14, lines 19-21 of Saito). The block combiner (figure 12(1111) of Saito) 
combines the blocks of each region (column 14, lines 1-7 of Saito) in order for said 
blocks to be output at the original input image resolution (column 14, lines 19-21 of 
Saito). In order to combine the blocks, said block combiner must allocate the pixels of 
the corresponding processed image. Otherwise, there will be no available memory to 
output the processed, block-combined image. 

Matsuda in view of Fields is combinable with Saito because they are from the 
same field of endeavor, namely image data processing. At the time of the invention, it 
would have been obvious to a person of ordinary skill in the art to use image reduction 
to analyze the image at a reduced resolution, as taught by Saito, said analysis being the 
show-through region analysis taught by Matsuda. Doing so would create an estimated, 
reduced-resolution show-through removed image which is then used to create the 
original image at the original resolution. The pixel allocation unit of Saito would be used 
to combine the reduced-resolution blocks and allocate the required data to create the 
corrected image at the original resolution. The motivation for doing so would have been 
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to reduce processing time and data storage requirements (column 14, lines 25-26 of 
Saito). Therefore, it would have been obvious to combine Saito with Matsuda in view of 
Fields to obtain the invention as specified in claims 10, 21 and 32. 

8. Claims 1 1 , 22 and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Matsuda (US Patent 6,285,470 B1 ) in view of Fields (US Patent 5,767,945) and 
Allen (US Patent 6,044,172). 

Regarding claims 11, 22 and 33: Matsuda in view of Fields does not disclose 
expressly a coordinate system transformation unit which transforms the color coordinate 
system of said original image or said reduced original image to another color coordinate 
system such as a YCbCr coordinate system or a pseudo KL color coordinate system, in 
which the components are highly independent from one another; and a coordinate 
system reverse transformation unit which performs transformation reverse to said 
transformation on the show-through removed image with respect to the image 
transformed by said coordinate system transformation unit. 

Allen discloses a coordinate system transformation unit (figure 1A(104A) of Allen) 
which transforms the color coordinate system of an image to another color coordinate 
system (column 4, lines 54-57 and lines 66-67 of Allen) in which the components are 
highly independent from one another (column 1, lines 13-18 and column 6, lines 39-42 
of Allen). 

Allen further discloses a coordinate system reverse transformation unit (figure 
1 A(106A) of Allen) which performs transformation reverse to said transformation on a 
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processed image with respect to the image transformed by said coordinate system 
transformation unit (column 5, lines 18-23 of Allen). 

Matsuda in view of Fields is combinable with Allen because they are from the 
same field of endeavor, namely image data processing. At the time of the invention, it 
would have been obvious to a person of ordinary skill in the art to convert the color data 
of Fields into a more convenient color space for processing the color image data and 
then reverse transforming said processed color image data, as taught by Allen, said 
processing being the show-through image removal taught by Matsuda. The motivation 
for doing so would have been to convert the color image data into a form that is more 
easily processed given the type of processing to be performed (column 5, lines 2-4 of 
Allen). Therefore, it would have been obvious to combine Allen with Matsuda in view of 
Fields to obtain the invention as specified in claims 1 1 , 22 and 33. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James A Thompson whose telephone number is 703- 
305-6329. The examiner can normally be reached on 8:30AM-5:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David K Moore can be reached on 703-308-7452. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 



Application/Control Number: 09/734,591 



Page 1 7 



Art Unit: 2624 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



James A. Thompson 

Examiner 

Art Unit 2624 
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